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ABSTRACT 

This study was conducted to assess the first year of 
full implementation of a teacher incentive program (TIP) in South 
Carolina. The study examines the relationship between student 
achievement and teacher participation in one of several incentive 
model programs. The research also explores award winning teachers and 
their association with higher gains by students in reading and 
mathematics achievement scores. A stratified random sample of schools 
was selected from which all TIP award recipients and a matched 
control sample of nonparticipants were selected as subjects, limiting 
the study to classroom teachers of reading and/or math in grades 1-6 
during the 1988-89 school year. Relevant characteristics for all 
teachers and their classes were recorded, along with achievement data 
for students. Results demonstrate that participants in TIP are 
associated with higher gain scores in reading and roith than 
nonparticipants, and suggest that screening candidates for teaching 
awards is consistent with the goals of performance-based compensation 
programs. (LL) 
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Introduction 



Incentive Systems for Teachers 

The decade of the 1980s has witnessed a remarkable resurgence of Interest 
In and concern about the nation's teaching force, demonstrated in part by the 
fact that virtually every state has enacted new policies governing teacher 
education, licencing, and compensation. Perceiving that their own state 
economies were at stake, -governors, business leaders, and legislators have led 
the charge in the "first wave" of teacher reform. In their efforts to reverse 
the decline of the quality of the teacher workforce, these decision makers 
ushered in new policies to: 1) assess the competencies of teachers through 
proficiency tests and tougher evaluations; and 2) provide incentives for 
attracting talented teachers through salary increases, merit pay programs, and 
career ladders. By the end of 1986, 29 states had initiated, piloted, or 
mandated alternative compensation structures through merit pay, career ladder, 
or other teacher incentive programs. 

State efforts to enact and implement performance compensation systems for 
teachers have been quite visible. Proponents claim that performance -based 
compensation systems will 1) meet the public's concern for teacher 
accountability; 2) enhance the status of teachers; 3) motivate teachers to teach 
better; and 4) attract and retain talented teachers. 

Several states - including Georgia, Florida, Kentucky, Alabama, Utah, 
Tennessee, and South Carolina - have taken the lead in mandating the use of 
student achievement data as a criterion for identifying superior teachers for 
incentive pay. Because of the assumption that improved student performance is 
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the "bottom line" in determining who is or who is not a "meritorious" teacher, 
measures of student achievement are a significant component in these states' 
teacher incentive programs. However, more often than not, states are having 
problems in implementing this component. For example, in Arizona, career ladder 
pilot districts are "backing off" on the student achievement component - 
primarily because school districts are "not comfortable that they have good 
measures" and they recognize that "it takes a while to standardize" the 
procedures for teachers and to "make it fair" (Olsen, 1987). 

It is important to note that performance -based compensation systems for 
teachers have come and gone for over 70 years. Historically, most of these 
systems failed within five years due to inadequate methods for judging 
meritorious teaching, administrative problems, and insufficient funding (Murnane 
and Cohen, 1984; Johnson, 1984). During the recent school reform movement, the 
greatest challenges to the implementation of performance -based pay plans have 
bein both technical and political in nature. 

The South Carolina Teacher Incen tive Proa ram 

As required by the Education Improvement Act (EIA) of 1984, the South 
Carolina State Board of Education hasi implemented an incentive program that 
"rewards teachers who demonstrate superior performance and productivity." In 
1985-86, the legislature appropriated $2.2 million, and three program models 
(Bonu; Model, Campus/Individual Model, and Career Ladder Model) were piloted in 
nin* school districts. In 1986-87, the legislature appropriated $6 million, and 
the Models were piloted in 17 school districts. In the third pilot year, 
1987-88, the legislature appropriated approximately $12 million, and the Models 
were tested in 44 school districts. The 1987-88 pilot- test was implemented in 



school districts which serve approximately 60 percent of the state's K-12 public 
school students. After pilot- testing, the Sonus and Campus/Individual Models 
were selected for statewide implementation. A brief description of the Bonus and 
Campus/Individual Models follows: 

Bonus Model 

To receive an award under the Bonus Model, candidates must demonstrate a 
superior record of performance in each of the following four areas : 

1. Attendance - Candidates must have an attendance record of no more than ten 
days of absence (out of 190-day contract year) during the year of 
consideration. 

2. Performance Evaluation - Candidates must be evaluated with a district 
instrument and obtain a rating determined through an appropriate standard- 
setting procedure to represent superior performance. Importantly, 
individual teacher performance must be "evaluated with a school district 
instrument which meets state criteria established by the State Board of 
Education for instruments used to evaluate Annual and Continuing Contract 
Teachers . " 

3. Self- Improvement - Candidates must dembnstrate evidence of self- 
improvement through advanced training. As a result, candidates are 
required to complete at least one self- improvement activity from the list 
of approved advanced training options. 

4. Student Achievement - All candidates are required to be evaluated for 
teaching performance as it relates to improved student learning and 
development. 
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Teachers meeting the requirements receive an award of not less than $2,000 and 
not more than $3,000 



Campus/Individual Model 

The Campus/Individual Model e: tablishes an incentive plan containing two 
separate components. The Campus Plan of the Model, which receives from one- third 
(1/3) to two- thirds (2/3) of the district's TIP funds, collectively rewards staff 
members in schools that attain superior levels of student achievement. Such 
schools will receive incentive funds to be distributed among eligible staff 
members. The intent of this approach is to reward eligible teachers in a school 
for working together as a team to positively influence student achievement. 
Under the Campus Plan, no eligible faculty shall receive an award which exceeds 
$3,000. 

The Individual Plan of the Model, which receives from one-third (1/3) to 
two-thirds (2/3) of the district's TIP funds, rewards individual school staff 
members for superior performance as detailed under the Bonus Model description 
without respect to overall school performance. Under the Individual Plan, no 
award shall be more than $3,000 or less than $2,000. For teachers who qualify 
under both Plans, no combined award shall exceed $3 000 . 

The initial three teacher incentive program Models were developed during 
the 1984-85 school year with the assistance of the Teacher Incentive Program 
Advisory Committee (TIPAC) and several consultants. TIPAC, together with the 
staff of the State Department of Education (SDE) , formulated criteria and 
procedures to fund a number of school districts for the development of model 
plans by October 1984. All school districts in South Carolina were then invited 



to submit proposals for developing model plans. Eleven school districts were 
selected by ihe State Board of Education to participate in the model development 
process, each receiving grants of up to $30,000 for the effort. 

Elements of the model plans submitted by the eleven selected school 
districts were reviewed and amalgamated into three program models by the SDE 
staff and TIPAC. These recommendations were reviewed by the Select Committee in 
December 1984 and approved by the State Board of Education in January 1985. The 
final selection of the three teacher incentive program Models was recommended by 
SDE staff and TIPAC, reviewed by the Select Committee and the Joint Subcommittee 
in April 1985, ^nd approved by the State Board of Education in May 1985. The 
three program Models encompassed exemplary features of the eleven district model 
plans in three distinctly unique formats. 

During the pilot- test phase, funds for the program were allocated to the 
districts on a formula (per pupil) basis. Funding for total implementation of 
the program was structured to enable up to 20 percent of each district's teachers 
to receive an award of $3,000. Twenty percent of the district's TIP monies or 
$3,000, whichever was greater, was to be used to cover operational costs. 

The three Models which emerged from the initial developmental process were 
the Bonus, Career Ladder, and Campus/Individual. Each Model was administered by 
a district-wide incentive committee with substantial teacher and principal 
representation. The committees set district procedures, screened applicants, 
approved each candidate's incentive plan objectives, and met with all approved 
candidates in an exit interview in order to evaluate evidence which verified 
attainment of approved incentive plan objectives. 

To be eligible for an incentive award, a participant must have been 
employed in the district prior to the initial application deadline as well as 



have achieved "professional statu*." For TIP purposes, candidates achieved 
"professional status" when they 1) were classified as a full-time employee; 2) 
were paid on the regular teachers' salary schedule (which means that guidance 
counselors, media spesialists, etc., were eligible); 3) achieved continuing 
contract status; and 4) had at least two years of teaching experience in South 
Carolina. 

At the conclusion of the first year of pilot- testing, the Models were 
revised with TIPAC's recommended changes for each of the Models. These 
recommendations pertained to teacher eligibility, program administration, and- 
procedures . 

At the conclusion of the second year of pilot- testing, the Models were 
revised to include TIPAC's changes and the Joint Subcommittee recommended to the 
State Board of Education that the Career Ladder Model be eliminated from the 
1987-88 pilot-testing. MGT of America, Inc., provided an assessment of the 
implementation of the 1987-88 pilot testing. In its most recent report MGT 
(1989) noted that: 

• 24 percent of the initial participants withdrew from the program due 
to the time and additional work required, 

• 78 percent of the remaining participants received an incentive 
award , and 

• a slight relationship was found between school gain indices and the 
percentages of t« achers who received individual incentiv* awards in 
the Campus/Individual Model. 

In 1988-89, TIP was implemented in each of the state's 91 school districts 

and three state agencies. During that year 23 school districts or state agencies 

implemented the Bonus Model while 71 districts implemented the Campus/Individual 

Model. Preliminary data indicate that, for the Bonus Model, 1,647 teachers 

participated - which represented 16 percent of the total number of teachers in 
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the 23 school districts. Of the 1,647 teachers who participated in the Bonus 
Model, 1,326 were awarded the $2000 to $3000 bonuses while 190 were disqualified. 
Of these 190 teachers: 

• 26 percent were disqualified because of the attendance criteria, 

• 4 percent were disqualified because of the self -improvement 
criteria, 

• 28 percent were disqualified because of the performance evaluation 
criteria, and 

• 42 percent were disqualified because of the student achievement 
criteria. 

With regard to the Individual Plan of the Campus/Individual Model, 3,558 teachers 
participated - which represented 13 percent cf the total number of teachers in 
the 71 school districts. Of the 3,558 teachers who participated in the 
Individual Plan, 2,494 were awarded the $2000 to $3000 bonuses while 776 were 
disqualified. Of these 776 teachers: 

• 12 percent were disqualified because of the attendance criteria, 

• 18 percent were disqualified because of the self -improvement 
criteria, 

• 12 percent were disqualified because of the performance evaluation 
criteria, and 

• 58 percent were- disqualified because of the student achievement 
criteria. 

With regard to the Campus Plan, 95 percent of the total number of schools (within 
the 71 districts) participated. Of the eligible teachers, 5,248 were awarded 
bonuses in the range of $208 to $3000. In addition, 1,091 teachers within the 
Campus Plan were disqualified. Of these 1,091 teachers: 

• 89 percent were disqualified because of the attendance criteria, 

• 1 percent were disqualified because of the self -improvement 
criteria, and 
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• 10 percent were disqualified because of the performance evaluation 
■ • criteria 

The Present Study 

The purpose of the present inquiry is to provide an assessment of the first 
year of the full implementation of the Teacher Incentive Program in South 
Carolina. The main issue before us is the following question: What is the 
relationship between students' achievement and their teachers' participation in 
the Bonus Model or the Individual Plan of the Campus/Individual Model? 
Specifically, are teachers who received an award associated with higher gains by 
students in reading and mathematics achievement scores? To attempt an answer to 
this difficult question, we have selected a stratified random sample of schools 
from which all TIP award recipients and a matched control sample of 
nonparticipants have been selected as subjects, limiting the study to classroom 
teachers who taught reading and/or math in grades 1-6 during the 1988-89 school 
year. Relevant characteristics for all teachers and their classes have been 
recorded, along with achievement data for students. A more detailed analysis of 
data collection and statistical analysis follows in the next two sections. 

Sample Selection and Data collection 

Sample Selection 

The sample selection technique employed in the present study is a multi- 
level stratified random sampling of districts and schools. In the- first stage, 
22 school districts were randomly selected, 14 districts participating in the 
Campus/Individual Model and 8 participating in the Bonus Model, using a random 
number table. To insure that the largest districts in the state would be 
represented in the study, the school districts of Charleston, Greenville and 

• 12 
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Richlind 1 were forced into the sample. In the second stage, again using a 
random number table, 70 schools were selected from the 14 school districts using 
the Campus/Individual Model and 57 schools were selected from the 8 districts 
using the Bonus Model. The matched sample was selected on the basis of the 
following characteristics: grade level, field of teaching, age, sex, race, 
educational preparation, and teaching experience. 

To ascertain that the sample districts were representative or the entire 
state, we compared attributes of the 22 districts to the state as a whole. 
Variables used in the analysis were the dropout rate, average years of experience 
of the teaching staff, percentage of the teachers who are male, average salary 
of the teaching staff, and assessed valuation per pupil of real property. Data 
yfor these variables suggest that our sample is representative of the state as a 
'whole since none of the t ratios are statistically significant at p - 0.05. 

In addition, the selected districts were located on a South Carolina map 
to assure that the sample districts weie chosen from each of the Congressional 
districts and each of the major regions of the state. 

Data Collection 

The SDE notified by letter each school district selected for the sample. 
Several of the districts were contacted by telephone and preliminary interviews 
were scheduled with selected TIP coordinators or other district representatives. 
Each school district was then contacted by telephone to request cooperation and 
explain the nature oi the research as well as the data required from the school 
districts. Letters were also sent to selected school superintendents to verify 
the data requested by telephone. In addition, two tapes of student achievement 
scores were obtained from the SDE - the second tape included teachers names for 
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the school district which indicated that achievement tests for elementary 
students were administered by the students' classroom teacher. 

As data vere received either from the school districts, the schools, or the 
teachers, the class rolls were matched to students' names from the tapes of 
achievement scores, with the exception of one school district. In the latter 
case, teachers' names were matched to the test administrators' names. 

Statistical analysis 

Gain Scores 

The first step is to compute gain scores for each student in our sample. 
Using the regressions in Table 1 (provided by the S.C. Department of Education), 
we computed the following: 

(1) P(Rq) - b xrg Rg- x + b 2r(J Mg. x + b 3r gRg. x 2 + b 4rg Hj- x 2 + b 5rg RMg- x + C rg 

where P(Rg) is the predicted value of the reading achievement score in grade g 
for the student basev? on his/her performance in the prior year (grade g-1) 

b irg is the relevant (ith) regression coefficient for the reading equation 
in grade g - 1 (from Table 1) 

• R g . x and M 8 . x are the student's actual reading and math scores in the 
prior grade, respectively 

R 2 and M 2 are the respective squares of the scores 
C rg is the constant term (also from Table 1), and 
RM,^ is given by the following formula: 

(2) RM g _ x - (R g - X - R,, g - X ) x (M s . x - M 8ig . x ) 

where R, (g . x and M i(g . x , respectively, are the state means for reading and math 
scores in the prior year. The state means are shown in Table 2. 
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TABLE 1 



RESULTS OF IMP AND RMP REGRESSION ANALYSES BY GRADE 
Dependent variable is achievement score on CTBS for grades 4 and 5, and 
SC Basic Skills Assessment Program for grades 1-3 ana b 



Co. st. Mult RSQ RMSE 




IMP 
RMP 

IMP 
RMP 



R 
M 
R 
M 

R 
M 
R 
M 



5.9147* 
4.3156* 



NA .0421* 
NA .0036* 



IMP 
RMP 

IMP 
RMP 



R 
M 
R 
M 

R 
M 
R 
M 



IMP R 
M 

RMP R 
M 



IMP 
RMP 



R 
M 
R 
M 



No Analysis Conducted 



.6147 
.3422 
.5715 
.1426 



.0865 
.2169 
.1428 
*. 3205 



.0010 
.0003 



.0004 
. 0009 



,0007 
.0004 



258.744 
392 . 644 



224.686 
353.127 
287.039 
479.610 



.2646 
.2716 



.4164 
.2476 
,3000 
.1435 



80.4990 
77.4501 



72.1120 
74.7959 
71.7342 
75.3379 



,5243 
.3614 
.5838 
,1140 

.4308 
.1332 
.6565 
.0521 



1548 
,3827 
,0813 
,4776 



.0011 



.0008 



1233 -.0012 -.0001 
,1151 -.0003 -.0002 
.1858 .0011 
.4716 .0001 .0008 



.0006 
.0010 



,0006 
.0003 



252.843 
204.288 
295.204 
368.876 

240.627 
487. -.93 
127.762 
335.938 



.4391 
.2784 
.3050 
,1985 

5106 
.4331 

5214 
,3651 



64.1392 
93.0775 
62.0800 
81.7751 

42.2937 
22.6734 
38.9423 
19.9672 



.6800 
.0936 
.7157 
.0664 



.2249 
.4376 
.1892 
.4777 



.0003 



.0016 .0008 
.0001 .0006 



87.143 
334.494 

82.742 
324.741 



.7255 
.6022 
.5784 
.4074 



27.9840 
'13.9365 
31.2937 
14.5323 



.9673 .9059 .0014 

.5310 2.6360 - .0101 

.5117 .1901 .0003 

.2041 .6113 - .0003 



-551.653 
■1455.548 
233.051 
156.277 



.6785 
.5413 
.3634 
.3546 



56.8222 
74.1410 
58.2233 
69.3094 



NOTE- R and M refer to reading and math scaled pre- test scores R and ^M are 
their squares (corrected to the mean) and RM is their product A ^dash ( ) 
in a cell indicates that the criteria for inclusion were not satisfied NA 
locates not appropriate. RMP indicates ^£ £ 

^T^^ 5 refers to CSAB raw 

scores . 



Source: 



S. C. State Department of Education. 
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TABLE 2 

Means to be Used for Deviation Terms (Scores from Spring 1987) 



Non-Repeaters 








Math 


H<5AR 
\jOixD 


94 3427 




1 
J. 


844 834 


805 025 

U V V • W Mm ^ 


0 
& 


811 757 


814 477 


3 


799.015 


806 851 


4 


679.721 


684.378 


5 


703.607 


698.120 


6 


776.892 


768.796 




Repeaters 




Prior Year Grade 


Reading 


Math 


2 


670.748 


699.768 


3 


689.887 


686.444 


4 


598.363 


643.168 


5 


637.640 


669.330 


6 


685.554 


667.071 



Source: S.C. Department of Education 

Predicted scores for math achievement are measured in a similar fashion 
as shown in Equation (3): 

(3) P (M.) - b^R,-! + b^M,^ + ba^R,-! 2 + b 4Bf M,_i 2 + b W*Vi + 

where the subscript m refers to the variables and coefficients associated with 
math. Note that the coefficients for some of the right-hand side arguments are 
sometimes zero, in which case these items obviously disappear from the equation. 
The gain scores are measured as residuals, as shown in Equations (4) and 

(5): 

(4) G gr - (Rg - P(R,)] x 10 / MSE gr 

(5) Gpi - [Mg - P(M,)] x 10 / MSEgj, 

where G gr and G^ , respectively, are reading and math gain scores, and 

MSE g „ and MSEga are, respectively, the mean squared errors for reading and 
math, shown in the rightmost column of Table 1. 
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L®t G »rij be tne reading gain score for the ith student in grade g taught 

by the jth teacher. Then we derive an average gain score for each teacher j, 

given by AG |rJ - lG gEij /nj, where nj is the number of students taught by teacher 

J 

J. An alternative measure would be the median gain score for the jth teacher, 
call it MG |rJ . Average and median gain scores for math (AG^ and MG^, 
respectively) are derived in a similar fashion. 

The principal purpose of calculating these standard residualized gain 
scores is to facilitate analysis in which gain scores may be used for all grades 
together. This could not be done with raw scores or simple gain scores. 

Cross Tabulations 

Tables 3 and 4 provide a tabulation of average reading and math gain 
scores by grade and by teacher group. The number of observations (teachers) in 
each cell is provided along with the mean gain score and the respective standard 
deviation. 

TABLE 3 

Number of Observations (N) , Means, Standard Deviations (SD), 
by Teacher Participation Group, of Average Reading Gain Scores, by Grade 



G NONPARTICIPANTS AWARD WINNERS 

R 

A CAMPUS MODEL BONUS MODEL 



D 
E 


N 


MEAN 


SD 


N 


MEAN 


SD 


N 


MEAN 


SD 


1 


28 


1.56 


4.76 


35 


3.39 


5.06 


38 


1.79 


4.78 


2 


37 


-2.03 


5.43 


28 


•0.43 


5.77 


26 


2.80 


3.28 


3 


30 


•3.96 


13.49 


38 


•1.10 


12.92 


25 


2.51 


5.79 


4 


30 


•3.46 


8.25 


21 


-0.07 


5.81 


31 


-2.22 


4.69 


5 


21 


1.43 


3.89 


11 


1.54 


3.20 


27 


1.43 


4.82 


6 


9 


2.38 


3.39 


7 


1.28 


3.09 


21 


6.17 


4.09 
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TABLE 4 

Number of Observations (N) , Means, Standard Deviations (SD), 
by Teacher Participation Group, of Average Math Gain Scores, by Grade 



G 
R 
A 
D 
E 




NlNPARTICIPANTS 






AWARD 


WINNERS 






N 


MEAN 


SD 




CAMPUS MODEL 






BONUS MODEL 


H 


MEAN 


SD 


N 


MEAN 


SD 


1 


28 


2.35 


4.65 


35 


2.23 


5.51 


38 


2.68 


4.43 


2 


37 


-1.30 


5.02 


28 


1.06 


6.28 


26 


4.17 


4.17 


3 


30 


0.41 


7.47 


38 


1.92 


11.36 


25 


2.65 


6.10 


4 


30 


-0.75 


6.85 


21 


0.90 


8.36 


31 


0.40 


4.02 


5 


21 


1.67 


3.91 


11 


2.00 


4.20 


27 


2.34 


3.57 


6 


9 


2.07 


6.16 


7 


4.04 


4.79 


21 


8.52 


5.70 



Reading 

Concerning readings scores, we note that the means of average gain scores 
(Table 3) are higher for both the B (representing the Bonus Model) and C 
(representing the Campus/Individual Model) groups relative to the NP (control) 
group. However, there are a number of cases where the reverse is true or where 
the differences are rather small. Moreover, because of the relatively small 
sample sizes combined with the relatively large standard deviations, most of the 
pairwise comparisons were not statistically significant. Similar results were 
found for median gain scores. 

Math 

Whether one uses the average or the median gain scores (see Table 4 for 
average gain scores), mean gain scores are typically higher for both the B and 
C groups relative to the NP group. In addition, the B group tends to have higher 
mean gain scores more often than the C group. Again, the number of observations 
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is relatively small and the standard deviations relatively large, hence we do not 
find many pairvise comparisons to be statistically significant. 

REGRESSION ANALYSIS 
The Model 

As noted earlier, the basis for the South Carolina Teacher Incentive 
Program is the assertion that "deserving" teachers should receive a financial 
reward. One of the cornerstones of the program is to reward teachers whose 
students gain relatively more in reading and math achievement. If the program 
works according to its design - which many teachers and other critics of TIP 
challenge - then we should expect to find a significant association between 
participation in the program and gain in achievement. In addition, the 
educational production function literature (for surveys see, e.g., Hanushek 
[1986], Monk [1990], and Cohn and Geske [1990], ch. 7) suggests that other 
classroom and teacher variables (among others) might affect reading and math 
performance* Tnese variables are included in Equation (6), which is our basic 
regression model: 

(6) AG - a + b x B + b 2 C + b 3 SEX + b*RACE + b 3 EXP + b 6 BA + b 7 BAPLUS +b 8 MA 
+b 9 GRADE + b l0 CSIZE + b u PCTBLACK + b 12 PCTFEMALE + b 13 PCTFRL 

where 

a- intercept; 

B- 1 for award winners in the bonus model and 0 otherwise; 
C- 1 for award winners in the campus/individual model and 0 
otherwise; 

SEX-1 for female teachers and 0 otherwise; 
RACE-1 for black teachers and 0 otherwise; 
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EXP- number of years of teaching experience; 

« 

BA - 1 for teachers with a Bachelor's degree only and 0 
otherwise; 

BAPLUS - 1 for teachers with BA plus 18 semester hours and zero otherwise; 

MA - 1 for teachers with a Masters degree and o otherwise; 

GRADE- grade taught by teacher; 

CSIZE- number of students in the class; 

PCTFEMALE- percent of students in class vho are female; 

PCTFRL- percent of" students in class who are eligible to receive either 

free or reduced- fee lunch; 
b t - b 13 are coefficients to be estimated by ordinary least squares 

analysis . 

The principal hypotheses are that b x > 0 and that b 2 > 0. One could also 
hypothesize that b 13 < 0, because poor students are often academically 
disadvantaged. 
Results 

The regression model was run four times: 1) for average gain scores in 
reading, 2) for median gain scores in reading., 3) for average gain scores in 
math, and 4) for median gain scores in math. Regression results are reported in 
Table 3. 

The most consistent result shown in Table 5 is that, other things equal, 
teachers receiving an award in either the Bonus or the Campus/Individual models 
appear to be associated with students having higher average gain scores compared 
to nonpar tic ipants. The size of the regression coefficients (varying from 2.21 
to 2.79 for reading and from 1.89 to 2.71 for math) as well as the t statistics 
strongly suggest that our results are important both quantitatively and 
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TABLE 5 

Estimated Partial Regression Coefficients (b) 
and t Statistics 

Dependent Variables: Average or Median Gain Scores 

in Reading and Math 



DEPENDENT VARIABLES 
AVERAGE GAIN SCORES MEDIAN GAIN SCORES 

Reading Math Reading Math 



INDEPENDENT 
VARIABLES 


b 


t 


b 


t 


b 


t 


b 


t 


INTERCEPT 


1.96 


.63 


2.07 


.65 


5.12 


1.89 b 


3.13 


1.03 


B 


2.43 


3.49 a 


2.79 


3.96 a 


2.21 


3.67 a 


2.71 


4.01 a 


C 


2.64 


3.69 a 


1.89 


2.61 a 


2.35 


3.81 a 


2.26 


3.26 a 


SEX 


.14 


.06 


.84 


.33 


- .89 


- .41 


.42 


.17 


RACE 


1 .06 


1.04 


- .98 


- .95 


1.45 


1.64 


-1.23 


-1.24 


EXP 


- .03 


- .66 


- .06 


-1.14 


- .02 


- .56 


- .06 


-1.24 


BA 


-1.86 


-1.70 b 


1.46 


1.33 


-2.76 


-2.93 a 


.97 


.92 


BAPLUS 


.12 


.10 


1.91 


1.53 


-1.41 


-1.31 


1.51 


1.27 


MA 


.08 


.08 


1.96 


1.98 a 


-1.00 


-1.18 


1.49 


1.57 


CSIZE 


- .05 


-1.26 


- .07 


-1.71 b 


- .01 


- .37 


- .06 


-1.66 b 


GRADE 


- .18 


- .91 


.11 


.55 


- .29 


-1.71 b 


- .09 


- .48 


PCTFEMALE 


4.04 


1.64 


- .20 


- .08 


3.08 


1.44 


1.19 


.50 


PCTFRL 


-6.63 


-5.54 a 


-5.18 


-4.27 a 


-8.55 


-8.28* 


-5.61 


-4.82 a 


R 2 


0 


.16 


0 


.12 


0 


.26 


0 


.14 


F 


5 


.66 a 


4 


.OO 3 


10 


.15 a 


4 


.82 a 


N 




364 




364 




364 




364 


NOTES : 



a Statistically significant at the 5 percent level, two-tailed test. 
b Statistically significant at the 10 percent level, two- tailed test. 
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statistically. In essence, these results suggest that the TIP program appears 
to have selected the "right" teachers, consistent with the intent of the program. 
It is also interesting to note the following results: 

1. Teacher's gender does not have a perceptible effect on achievement gain 
scores . 

2. Teacher's race has a small, primarily nonsignificant impact on 
achievement gain scores, though the effect of race has opposite effects on math 
and reading scores (black teachers seem to have a small edge in reading but a 
negative edge in math) . 

3. The coefficient of EXP is uniformly negative, though not very large and 
also not statistically significant even at the 10 percent level. We also 
reestimated the model with both EXP and its square (EXPSQ) , but the coefficients 
of EXPSQ are uniformly nonsignificant. 

4. In assessing the effect of education (BA, BAPLUS and MA) on gain 
scores, it must be remembered that the coefficients in Table 5 are to be compared 
to the excluded category, which is the level of education beyond MA (call it 
MAPLUS) . Our results suggest that teachers with a BA only are associated with 
significantly lower reading gain scores in comparison to all others, but that 
teachers in the BAPLUS or MA categories have reading gain scores roughly similar 
to those with MAPLUS. Also, the excluded group, MAPLUS, appears to have lower 
math gain scores compared to all other groups, especially the BAPLUS and MA 
groups, whereas no significant differences are found among the BA, BAPLUS, and 
MA groups. 

5. We also tested the proposition that experience and education interact 
in the equation. We formed three new variables, as follows: 

BAX - BA x EXP; 



9 

ERIC 



22 



19 

BAPLUSX - BAPLUS x EXP; and 
MAX - MA x EXP. 

In general, the Interaction terms were not significant, and other regression 

results were not materially altered. Also, F tests (Johnston, p. 146) indicate 

that in none of the equations do the three new variables add a significant 

contribution to the explanatory power of the model. However, results for mean 

reading gain scores produce the following equation (t-values in parentheses): 

MREAD - 5.34 + 2.33 B + 2.59 C + 0.08 SEX + 0.85 RACE - 0.24 EXP 
(1.40) (3.33) (3.64) (0.03) (0.84) (-1.82) 

-6.66 BA - 3.59 BAPLUS - 2.72 MA + 0.37 BAX + 0.24 BAPLUSX + 
(-2.68) (-1.09) (-1.12) (2.27) (1.21) 

0.17 MAX - 0.05 CSIZE - 0.20 GRADE +5.23 PCTFEMALE - 
(1.17) (-1.29) (-1.04) (1.71) 

6.49 PCTFRL 
(-5.43) 

(R 2 - 0.17. F - 4.96, N - 364) 
These results suggest that additional years of experience have a net negative 
effect on reading gain scores for all teachers except those with a BA degree 
only, for whom the effect is slightly positive (.37 - .24 - .13). Also the 
marginal effect of BA on reading gain scores is -6.66 + .37 (EXP), suggesting 
that after 18 years of experience, the effect of BA on achievement is positive. 

6. Class size has a negative coefficient throughout, and the effect is 
significant at the 10 percent level for both mean and median math gain scores. 
A negative sign suggests that r.aaller classes are better, a result that is 
plausible yet not frequently observed. 

7. With one exception, grade level has a negative effect (i.e., gain 
scores decrease at higher grades), but (again with one exception) the results are 
not statistically significant. 
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8. Classrooms with a larger percent of females tend to do better, though 
the results are not statistically significant. 

9. As expected, gain scores are much lover in classrooms with a higher 
percentage of poor students. This result is consistent with the Coleman Report 
and many other studies emphasizing the effect of socio-economic status on student 
achievement , 

CONCLUDING COMMENTS 

Our results clearly demonstrate that participants in TIP are associated 
with higher gain scores in reading and math than nonparticipants , This suggests 
that, at least in part, the screening by committees of candidates for a teaching 
award is consistent with the goals of the program. It is impossible to tell 
whether our regression results suggest causation - i,*s,, that award winners 
influence their students ' gain scores - or whether the results merely suggest an 
association - i.e., that the award winners simply happen to have students who 
have higher gain scores, and that various factors not analyzed here might play 
an important role (Berk, 1990). Our guess is that there is at least some 
causation here, although how much we can obviously not tell without further 
study. 

One must view our results with appropriate caution. First, although we 
have controlled for "self selection" by introducing a number of teacher and 
classroom variables and by a judicious selection of the sample of nonparticipants 
(the control group), there might still be some latent self -selection effect of 
which we are not aware. Second, although the sample size is adequate for 
analyses of this type, a larger sample would clearly increase our confidence in 
the results. Finally, as is true in all empirical analyses, errors in 
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measurement and specification are unavoidable, so that replication is required 
if one is to have confidence in the empirical results. Although a study of this 
type is costly both in state funds a. i the contribution expected of local school 
districts, we believe that replication over time is necessary. 
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